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FOREWORD 



This study of National Assessment and Social Indicators is one 
of a series of exploratory efforts under OEC Contract 0-70-4454 tc 
examine and report on educational outcome measurements. 

In this report a series of questions is considered: What does 
National Assessment contribute to measurement of the quality of life? 
What changes in National Assessment of Educational Progress dc*ta would 
improve the base for understanding educational achievement? How might 
National Assessment of Educational Progress data be applied as social 
indicators? In any case, what changes might be made to gadn more wide- 
spread use of the National Assessment materials for a broader range of 
purposes? 

As a preliminary step toward examining these issues 5 the present 
report outlines the types of findings on educational achievement that 
might be made from the statistical data on knowledge, attitudes and 
skills being gathered. Further, social indicators might be developed 
from the data on knowledge in 10 subject matter areas. Among the 
innovative ideas set forth is the possible use of National Assessment 
for an educational product index (EPI) that woxild be applied much in the 
same way as the Consumer Price Index (CPI) — an index of change over time. 

^ While we may have doubts about broadening the scope of National 
Assessment to serve purposes beyond specific educational assessment 
needs ^ the question posed regarding social indicator applications 
certainly warrants consideration. The: Office of Education therefore 
believes "this document deserves wider dissemination and discTission. 

Dorothy M. Gilford 
Assistant Commissioner for 

Educational Statistics 
U.S. Office of Education 
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PREFACE 



For readers unfamiliar with the National Assessment of 
Educational Progress, some background information may be helpful 
before beginning section 1. 

National Assessment is a survey of what U.S. citizens know 
and can do in 10 broad subject areas. It has been conducted since 
1969 by the Education Commission of the States and has been sup- 
ported primarily by the U.S. Office of Education. • 

Exercises at each of four ages — 9, 13, 17, and young adult — 
range from very easy to very difficult. Results of some exercises 
are released in periodic reports for educators and the public; 
other exercises are held back for reassessment to indicate change 
over time. 

Each person in the sample takes a portion of the exercises. 
Individual results are not released; however, differences in sex, 
region, size of city,, parental education, and other variables are 
analyzed to assist educational planners. 

Although National Assessment does not directly measure 
health status and other social indicators, it does measure the 
skills J understandings, and attitudes which are necessary for 
behavior which affects these indicators . 

For additional information on National Assessment, see: 

Frank B. Womer, What ±s National Assessment? Ann Arbor, 
Michigan: NAEP, 1970. 

Carmen J. Finley and Franoes S- Berdie. The National 

Assessment Approach to Exercise Development . 
Ann Arbor, Michigan: NAEP, 1970. 



ERIC 



- iv 



TABLE OF CONTENTS 



Page 



Acknowledgments Inside Frunt yover 

Foreword iii 

Preface iv 

I. National Assessment Data as Social Indicators 1 

Toward Social Measurement .1 

The National Assessment Exercises--A Step 

Toward Social Indicators 2 

Findings of the National Assessment as Social 

Indicators , 10 

II. National Assessment of Educational Progress 11 

What have we learned from National Assessment?.. 12 

What do the findings show about our cumulative 

knowledge as a nation? 12 

What more could we learn? IH 

Are we learning more over time? 15 

What kinds of knowledge di.d everyone (or nearly 

everyone) have? .16 

Are reading scores a good proxy? .16 

Does the education of parents pose the same 

hurdle to educational attainment in each of 

the learning areas? 17 

Is the retention of knowledge sufficient to 

indicate that existing methods of preparatory 

education are efficient? .17 

On what types of exercises was there almost 

uniformly incorrect response? 18 

III. An Agenda for the F"ture. , 18 

Social Reporting as Part of National Assessment .18 

Improving Measures and Application of Measures 

of Educational Progress ...20 

Charts and Tables 29 

Appendix Tables 41 

References in Text 51 

Additional References 52 



r 



LIST OF CHARTS, TABLES AND APPENDIX TABLES 

Chart 1 — Male -Female Edge of National Assessment 

Tests by Subject arid Age 30 

Chart 2 — Median Male -Female Differences for Physical and 
Biological Science Exercises of the National 
Assessment 31 

Chart 3 — Edge for those whose Parents had Post High School 

Education over those witii no High School Education ... .32 

Chart ^^ " White Edge— by Subject and Age 33 

Table 1 — National Median Percent Acceptable Answers on 

National Assessment Exercises 34 

Table 2 — Median Percent Acceptable Answers for the 

Northeast and Southeast Regions on National 

Assessment Exercises 35 

Table 3 — Median Percent Acceptable Answers by Race on 

National Assessment Exercises 36 

Table M- Median Percent Acceptable Answers for Rural and 

Inner-City Areas on National Assessment Exercises 37 

Table 5 — Median Percent Acceptable Answers for Males and 

Females on National Assessment Exercises 38 

Table 6 — Median Percent Acceptable Answers on National 

Assessment Exercises by Parental Education — Post 

High School Education and No High School Education 39 

Table 7 — Reading Achievement by Age and by Theme HO 

Appendix Table 1 — Sample Population Categories 42 

Appendix Table 2 — Timetable for Assessment Exercises 43 

Appendix Table 3 ~ Educational Objectives in Five Subject 

Areas 44 

Appendix Table 4 — Number of Exercises by Type 46 



ERIC 



- vi 



NATIONAL ASSESSMENT AtCD SOCIAL INDICATORS 



Charles Dickens^ Mr. Gradgrind would welcome the National Assess- 
ment of Educational Progress. Social measurements, Mr. Gradgrind firmly 
held, could solve any social problems; the basic requirement for 
resolution was quantification* Certainly the National Assessment is 
designed to be the basic meabcrement of educational achievements and 
progress. Knowledge, understanding, skills, and attitudes of the 
American people are counted. 

What does this assessment contribute to measurement of the quality 
of life or, conversely, social ills? This paper addresses this broad 
question, asks specifically what purposes are now served and how they 
can be served better, and then presents an aganda for review. 



I. NATIONAL ASSESSMENT DATA AS SOCIAL INDICATORS 



Toward Social Measurement 



Up to a few years ago the basic yardsticks of progress in the 
United States were economic. Changes in per capita income and unemploy- 
ment rates represented primary measurements . But for in^o groups with 
the same per capita income and unemployment rates, levels of welfare 
clearly differ if one group is healthy and the other sick. Their level 
of well being differs, too, if one feels safe in the street and the 
other unsafe. Or it differs if one group has a depth of knowledge about 
history, scientific relationships, and cultura,l patterns, while the other 
is ignorant and .Mliterate and isolated. It was to achieve a quantitative 
reflection of thase differences in levels of well being, or quality of 
life, that social measurements were designed. Whatever the caption, the 
intent is to enlarge the political arithmetic and provide signals of a 
growing weakness or strengthening of society. 

By 1972, the terms "social indicator," "social accounts," "social 
bookkeeping," "social reporting of social change," have been widely used 
to identify social quantification. 

We will not linger on the history, definition, or present status 
of social measurement. Such measurements may be traced to Petty (1) and 
Quesnay (2); in recent years Bauer (3), Bell (4) , Biderman(5), Duncan (6), 



Gross (7) 3 Land (8), Moore (9) , Olson (10) and Sheldon (11) contributed 
significantly to the literature, including reassessment of claims made 
for purposes of social indicators. 

Thus social indicators on the quality of life are being formulated. 
Although they are still in the pioneering, experimental stage, work to 
date has provided a useful tool for identifying and clarifying directions 
in what often appears an impeneti?able tangle of conflicting facts, 
judgments, and assertions. By marshalling the quantitative data, putting 
them into an appropriate format for wide distribution, and rendering 
them comparable, social indicators can give policy officials a more 
intelligible and comprehensive view of the quality of life. This is the 
message of Toward a Social Report ri2) , a report to the Secretary of HEW 
which represents a preliminary step toward a system of social reporting. 

The starting point for discussion of National Assessment as a 
data base for social indicators is the enumeration of relevant aspects 
of quality of life. Some of the thorniest conceptual issues arise in 
the identification and selection of indicators and the systems for which 
those indicators become inputs. 

Models of social accounts must grapple with theoretical problems 
at the frontier of social science knowledge. Even for economics, the 
harder of ^ the social sciences, there are conceptual questions about 
consumer theory, the interrelationships between public and private goods, 
and the proper way to handle the redistribution impacts or inequalities 
of income, opportunity, and wealth. For this paper we have not attempted 
any analysis of appropriate measurements. Rather we have restricted our 
review to Nationpl Assessment, looking particularly at health, equality 
of opportunity for blacks and women, safety and puilic order, and 
democratic participation and alienation. 

The National Assessment Exercises — A Step Toward Social Indicators 



Data collected by the National Assessment provided indirect in-- 
dicators of educational achievements. Can we also obtain indicators 
from those data that measure more general aspects of social life? For 
example, can we get some insight into health habits, attitudes toward 
public order and safety , equality of opportunity for woT^en and blacks? 
Can we learn about the extent of alienation? And to what extent can data 
be obtained about other social measurements? 

The Education Commission of the States selected ten subject-matter 
areas for study. They include the basic skills of the 3 R^s — reading, 
writing, mathematics — as well as literature. The remaining six areas- 
art, music, social studies, science, citizenship, and career and 
occupational development — cover aspects of our social life. 
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Results are now available for five of the ten learning areas ; 
two of them contain exercises that could be usefully applied to social 
measurements: science and citizenship. We draw on the exercises from 
science and citizenship to document the potential use of educational 
assessment data for social indicator measurements, and suppl^^inent this 
documentation with potential exercises for career and occ ipt^tional 
development, an area now being assessed. 

Between citizenship and science, perhaps 50 exercises are most 
immediately applicable as items of social assessment (that is, measuring 
some aspect of the quality of life other than a more narrowly defined 
output of the educational process). Until the listing of the questions 
in the exercises has hern completed, we cannot suggest the full range 
and scope of possible social indicator uses of the data. 

A specific detailed example may clarify the process and adapta- 
tion involved, and illustrate the questions that arise in using National 
Assessment data as social indicators. Toward a Social Report (12) in- 
cludes a brief section on freedoms and gaps in measurements of freedoms. 
Without claiming that the freedoms in their many dimensions have been 
defined in a way that would adapt easily -o measurement, or that we know 
how best to collect the data on freedoms, we note that the National 
Assessment includes exercises to assess attitudes on freedoms. 

Among the objectives defined in assessing education's role in 
citizenship is the objective to ^Icnow the main structure and functions 
of our government." And within that larger function is the subobjective 
"recognize the main functions and relations of governmen*'al bodies ; to 
recognize that governmental authorities are limited by the people through 
the Constitution." One exercise foi* this subobjective is: "Does the 
President have the right to do anything affecting the United States that 
he wants to do?" The answers "yes," "no," and "I don^t know" are listed 
along with subexercises indicating that if the initial question was 
answered "yes," explain why, and alternatively If answered "no," 
explain why not . 

What do the answers suggest about the knowledge and views about 
Constitutional restrictions on Presidential authority? About half of the 
9-year olds answered "yes" in this exercise. For the young adults* 
sampled (those 26 to 35 years of age) all but 11 percent answered "no" 
correctly, but much smaller proportions gave acceptable reasons. Among 
the acceptable ways of responding listed by the National Assessment of 
Educational Progress are: 'people could stop him"; "elected officials 
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could stop him"; '^country would be a dictatorship"; "not the democratic 
way." Fewer than one out of five of. the Q'-year olds answered correctly 
with a ^'no" and could advance an acceptable answer to the question ox- 
why not. Even among young adults, one out of five could not advance 
an acceptable reason for a "no." 

Data in the National Assessment are classified by characteristics 
of respondents, as shown ih Appendix Table 1. What do the breakdowns 
indicate on restrictions of Presidential powers V First, the males ^ edge 
in correct responses over females increased with age. Not surprisingly 
the Northeast has more correct responses than the Southeast, and those 
whose parents have post -high school education are more often correct 
that those whose parents did not attend high school. These findings, 
discussed in detail later, suggest the types of social indicator data 
that can be obtained from the National Assessment. 

Thus perhaps 50 of the released exercises could be used as 
measures of quality of life, with these qualifications: 

- There is no specific link between research on concepts of social 
indicators and the formulation of objective exercises for the 
National Assessment. 

- There is no machinery to relate neasurements of social, psycho- 
logical, political views, attitudes, and knowledge to the tasks 
of objective setting and exercise formulation. 

- There is little understanding of the methodology required to gain 
accurate measurement of attitudes. 

- There are only piecemeal data on differences in attitudes toward 
responses on politically and culturally related questions. 

- There has been insufficient documentation of differences in 
striving for correct responses, or acceptability of incorrect 
responses to certain groups of respondents . (Difference may 
result in variations by age, sex, region, city size, parental 
education that reflect social conditions other than knowledge 
gaps.) 

Thus, some barriers to use of National Assessment as social 
indicators grow out of the insufficient cross -fertilization of ideas and 
research priorities between psychological, sociological and political 
scholars working on other measurements and those engaged in National 
Assessment. In addition, some barriers result from the Ijlational Assess- 
ment design: scope limitations or limitations of selected attitudes, 
skills and knowledge ''bits"; limitations that develop out of the process 
of objective setting, exercise formulation and reviews; and limitations 
that come .from educators * caution about political sensitivity to the 
queries and their uses . ^ 



Clearly, different persons would select different exercises for 
social indicator application from among those whose results have been 
published by National Assessment. We intend to be only illustrative 
rather than definitive, to give sufficient infoimation to define the 
possible uses of National Assessment for social indicator measurement. 
In Section III, we propose methodological changes that could better 
serve the purpose of social measurements through the National Assess- 
ment. 

Health Status oi rne Population . Health status is usually measured by 
negative data--by death rates or by morbidity rates. Sometimes life 
expectancies are computed from death rates by age. By combining death 
and morbidity rates, a new index was developed in Toward a Social Report — 
years of 'T.iving" within the years of life expectancy. Dunn and others 
have proposed an index of ^^-^^ellness" that would emphasize the positive 
aspects of health. Other measurements of health are "added working 
years" and "added earnings attributable to disease prevention or control." 

The National Assessment includes among its Science exercises nine 
that suggest skills, knowledge, and attitudes toward health. These are 
enumerated below by age of respondent, exercise number, and knowledge 



"bit. 



9-year olds : 



R 103 



teeth brushed to keep from decaying 
protein is employed in building good nuscle 
houseflies can spread serious dlsearcis 
recogni7.f» vaccination as the reason so few 
people get smaJ.lpox 



108 
134 
152 



13-year olds: 



207 



cancer is a disease that cannot at present 
be controlled by a vaccinij 

pasteurization of milk kills bacteria harmful 
to man 



219 



17-year olds : 



302 
306 



choose from alternatives the best balanced meal 
recognize that outlawing pesticides does not help 
to increase the total amount of fooc^ available 
to the human race 



Adults (26-35) 



R 417 



recognize that outlawing pesticides does not 
help to increase total amount of food available 
to the human race 

a malady that cannot be inherited is whooping 
cough 

recognize vaccination as the reason so few 
people get smallpox. 



404 



435 
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From the listing of objectives for career and occupational 
development, additional information on health habits and knowledge may 
be obtained under Objective IV: 'Practice effective work" habits," with 
the subobjective "to maintain good health and grooming." Eight specific 
subobjectives indicate view" and knowledge about health care. 

Age 9: Attend regularly to personal hygiene 

(e.g., bathe, wash hands before meals) 
Go to school health offices, dispen- 
saries, or doctors when sick or injured 
Stay at home when ill or when one has 
communicable diseases 

Age 13 (in addition to 9) : Refrain from use of harmful drugs 

Take precautionary measures to prevent 

illness C^-S-j exercise regularly; 

avoid unnecessary exposure; eat 

balanced meals) 
Take necessary steps to regain health 

during and after illness (e.g., take 

prescribed medication - rest - avoid 

overexertion) 

Age 17 (in addition to 13); Have regular medical and dental checkups 

Seek professional help when needed, for 
example , doctor, dentist, psychiatrist . 

Equality of Opportunity Measurements . Many measures of unequalness are 
used as social indicators. For those in poverty the indicators recognize 
that being poor catches the poor up in adversities that affect every 
turn in life and create an intergenerational cycle of poverty. The poor 
pay more, have less, purchase products of inferior quality, get locked 
into high-cost credit, receive in many places poor care, poor education, 
poor housing, so that "poorness" is the increasing effect of being poor. 

But other inequalities exist: for example, inequality in the 
school, or in certain employments for black persons or women. How are 
the differences in race, sex and religion viewed by respondents to the 
exercises and what is the understanding (knowledge and attitudes) about 
the causes of poverty and other inequalities? 

The National Assessment includes in its science and citizenship 
exercises nine that can thvow some light on attitudes toward equality 
and fairness. 



9-year olds: None 



R 245 believe that women can be successful scientists 
A 3 understand and oppose unequal opportunity in 
recreation 

A 4 treat all individuals with respect; do not 
condemn others on the basis of irrelevant 
personal or social characteristics, etc, 
A 5 aware of racial discrimijiation in the United 
States 

A 5 aware of religious discrimination in the United 
" . States 

B 4 believe some races nre better than others 

17-year olds: A 3 understand and oppose unequal opportunity in 

recreation 

A 4 willing to have a person of different race be 

a dentist or doctor 
A 4 willing to have a person of a different race 

live next door to you 
A 6 aware of religious discrimination in the United 

States 

Adults A 3 feel that they should pct to stop discriminption 

in public park 
A 4 treat all individuals with respect; do not 

condemn others 
A 6 aware of religious discriminetlon in the United 

States . 

Freedom and Participation , Among the major measurements used to indicate 
social change and current status of views on freedom^ participation, 
alienation, and other similar measurements have been voter participation, 
holding of public office, court cases on questions related to civil rights 
and personal freedoms, attitudes that indicate alienation. Little is 
known about appropriate measurements of freedoms; much work remains to 
be done to define the freedoms in a measurable way, and to capture those 
aspects of freedom that are defined. Voter counts and offices held by 
minority groups or by income class are more usual tools of assessing 
citizenship in a democracy. 

National Assessment states explicitly that its intent is not a 
"global" rating of overall quality of citizenship in respondents, but 
to measure achievement of particular citizensliip objectives. 

Nevertheless, in the National Assessment exercises on citizenship, 
32 questions provide data on measurements of general qualities of freedom, 
participation, or alienation. Although the resulting measurements provide 
a somewhat spotty profile of certain characteristics, they may serve as 
a beginning for a more complete set of social indicators* 



; 13-year olds : 
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9*year olds: B 1 know police do not have right to come inside 

one^s house, and give p reason 
B 5 think it is p11 right to tell other people thst 

the Governor or President is doing a bad job 

and give reason 
C 1 think we need rules on the playground and state 

re as on 

D 2 think the President does not have the right to 

do anything that he wants 
D 9 recognizs that deirocracy depends on alertness 

end involvement of its citizens 
E 9 actively work for community improvement - -e. , 

took part in special project to help other 

people or make the world a nicer place 
E 13 apply democratic procedures on a practical 

level when working in a group 
G 9 recognize that even when p newspaper says 

something is true it can be wrong 

13-year olds: B 1 (same as 9-year olds) 

B 2 know that a jury, trial, or court decides whether 
or not a person is guilty 

B 3 believe people should have right to assemble; 

would limit assembly when prohibited constitution- 
ally 

B ^^ think a person should have the right to express 
unpopular views on radio or TV (e.g., Russia is 
better than the U.S.) 

B 5 (same as 9 -year olds) 

C 2 name one or more reasons why laws are needed 
D 2 (same as 9 -year olds) 

D 3 recognize the importance of political opposition 
D 12 recognize the importance of diverse interest 
groups 

E 3 gave at least one reason wh;, hey might write 

elected officials 
E 7 willing to express own views on civic and social 

matters, however controversial the issue 
E 9 (same as 9 -year olds) 

E 14 apply democratic procedures on a practical level 

when working in a group 
F t gave one or more ways we can ^void future vjars 
G 5 believe education or training would help 

children from poor families have a chance to 

get better jobs 
G 9 (same as 9-year olds) 
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17-yepr olds: B 1 (seme ps 9-year olds) 
B 4 (same ps 13 -year olds) 
C 2 (SKTie as 13 -year olds) 
D 2 (spme as 9-yepr olds) 
D 3 (same as 13 -year olds) 

E 1 know one or more ways cit^'^^ns cpn influence 

actions of their government 
E 2 think they can have some influence on state 

government decisions Rnd gave f^t least one 

mepns of doing so 
E 5 wrote at least one letter to the editor of f 

newspaper 
E 7 (same as 13 -year olds) 

E 8 reported they have campaigned for a candidate 
E 11 seek out and participate in civic organization 

by holding office or serving on committees 
E 14 (same as 13 -year olds) 
F 4 (same as 13 -year olds) 
G 2,3,4,7 recognize important civic problems 

G 8 pware of facts and opposing arguments on current 
controversies 

Adults: B 1 (same as 9-year olds) 

B 4 (same as 13-year olds) 
C 2 (same as 13 -year olds) 

C 3 describe an unjust or unfair. law; report they 
h?*ve attempted to do something to try to get 
it changed 

D 3 (SRme as 13 -year olds) 

E 1 (same as 17 -year olds) 

E 2 (same as 17-year olds) 

E 4 reported they have talked or written to a 

governmental official on one or more issues 
E 5 (same as 17-year olds) 

E 6 reported they have spoken in a public meeting 

to uefend someone ox'' some idea 
E 8 (same as 17-year olds) 

E 10 reported they have taken part in one or more 

civic activities in past year 
E 12 belong to clubs concerned with community 

improvements 
F 4 (same as 13 -year olds) 
G 2,3,4,7 (same as 17-year olds) 
G 8 (same as 17-year olds) 
I 5,6 provide enviitonment that helps their children, 
to learn 
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Findings of the National Assessment as Social Indicators 

We earlier described the exercise on the powers of the President, 
and presented the findings in summary forn. Some of the findings from 
the exercises enumerated above are summarized further to underscore the 
potential applications of the National Assessment for social indicator 
purposes : 

- A larger percentage of young persons than adults felt that they 
should act to stop discrimination in a public park; the edge in 
answering correctly about actions that they could take went to 
persons in the big city and the 17-year olds- 

- A larger portion of 13 -year olds than of those 17 years of age, 
or young adults, would be willing to have a person of a different 
race as a dentist or physician. For the 13 -year olds, the great- 
est unwillingness was reported in the Southeast. 

* Three out of four adults indicated a willingness to have a person 
of a different race as dentist or doctor, but only two out of three 
were willing to have a person of a different race live next door. 
Again the Southeast reported the larger share of racial separation. 

- Only one out of five persons 13 years of age believe a person on 
the radio should be allowed to say "Russia is better than the 
United States." More adults believe that this statement should be 
allowed (56 percent) . 

- Only one out of about seven 13 -year olds believe that a person 
should be allowed to say on radio or TV that some races are 
better than others. The comparable percentage for the 17-year olds 
is almost 32 percent, and for adults 37 percent. 

- Only one out of about five 9-year olds knows that police do not 
have the right to come inside one^s house, and can give a reason 
why not. Almost one-third of the 13-year olds do not know, and 
almost one out of 5 adults . 

- More than one -third of the 13 -year olds are unaware of racial 
discrimination; fewer than one-half could cite an example; and 
only a little over one-fifth are aware of religious discrimination. 

- Just one-half of the 17-year olds and 59 percent of the adults 
stated they can influence state government decisions and gave a 
means of doing so. 

- A little over one-third of the adults reported that they have 
worked actively for community improvement. 

O 
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II. NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS 



Up to a few years j^go the emphasis in educational policy assess- 
ment was almost exclusively on educational inputs — resources used in 
the school. Except for literacy data, the educational stf^tistics on 
educational advance were confined to measures number of students 
enrolled at each educational level (e.g., high o^chool, college) or to 
expenditures for education. HEWs Toward a Social Report of 1969 (12) 
roted that The Digest of Educational Statistics contains over a hundred 
pages of educational statistics in each annual issue yet has virtually 
no information on how much children have learned. 

To remedy this deficiency, the first effort toward new measurements 
was to score achievements and use such scores as tests of quality. What- 
ever else education might produce, it was argued then (13), it surely must 
produce basic skills such as reading. This simplistic, sole reliance on 
test measurements such as reading scores originated in the thrust toward 
cost-effectiveness analysis in place of earlier non-quantified general- 
ities or input measurements. Despite the merit of the undexrtaking, some 
basic concepts of effectivness measurements were set aside in the drive 
to show the policy use of analytical studies. To question the emphasis 
on achievement measurement, principally reading test scores, was to 
question the- whole exercise — to question the value of ^Tiard" analysis. 

The National Assessment of Educational Progress originated in 
the drive to remedy the deficiency in earlier ^- ata. National Assessment 
was designed specifically as a data collection and analysis effort. To 
quote a recent report: 

...National Assessment has the purpose uf providing dependable 
information describing what young Americans (9, 13, 17-year olds 
and young adults 26-35) know and can do. More specifically the 
assessment is designed (1) to obtain, at regular, periodic 
intervals, census-like data on the knowledge, skills/ understandings, 
and attitudes possessed by various subpopulations in the United 
States, and (2) to nieasure the growth or decline in educational 
attainment that takes place over time in learning areas of 
educational concern. (14-) 

The timetable of National Assessment, reproduced in Appendix 
Table 2, indicateis that by the end of the academic ye^r 1972-73, eight 
of the ten learning areas will have been surveyed; two additional areas 
will remain: career and occupational development and art. Last year, 
data for the assessment of five learning areas were being analyzed 
simultaneously, and new data in two learning areas were being gathered. 
Scheduled for this current academic year (1972-73) is the first repeat 
of an assessment, science knowledge. 
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The National Assessment differs from the usual achievement testing 
in that it is not a test reporting indivi'ual scores to be aggregated. 
Total scores for persons are not obtained--no individual participates in 
more than a fraction of the testing. Ratheo.*, each respondent takes a 
package of exercises in which he is asked to demonstrate he has the 
knowledge, understanding, skill, or attitude for those exercises- 
Assessment exercises are designed to be easy, medium, or hard, to 
produce a gradient measure of knowledge and skills. 

Whereas achievement tests are usually normed, so that: individuals 
can be ranked or scored in relation to the norm. National Assessment 
yields information about how groups of persons answer individual 
exercises . 



What have we learned from National v Assessment ? 

From the series of exercises released for the five learning areas 
completed, we have learned what is known about those exercises at each of 
the dge levels for specific groups. We know the percent who have 
responded correctly to each of the exercises for which findings have been 
released. And we know those percentages for each of the selected charac- 
teristics — sex, coloz , city size, region, and parental edueal-.on. 

Comparisons are possible between age groups (when the same exercise 
is given to each age group), between sexes, between rural and urban areas, 
between regions, and between whites and racial minority groups. How 
similar or disparate is the knowledge about the information in exercises? 

When the followup science exercises are completed, for example, 
at the close of the year 1972-73, the. question can be answered: /How 
has knowledge changed over the five-year period?" Comparisons of the same 
exercise given in both years to the sampled population will yield specific 
data on change (if the sampling procedure is not altered so that sample 
biases are introduced). 



What do the findings show about our cumulative knowledge as a nation ? 

To date. National Assessment falls short of providing data on the 
nation *s knowledge for macro planning of education. First, its scope is 
restricted to young respondents; what we as a nation know in the 
preparatory years is more nearly the scope of the National Assessment. 
Second, a weighting process has not been developed for all the subject 
areas to summarize knowledge gained in each of those subjects. The 
concept of the National Assessment is to have an objective -based 
system rather than a norm-referenced system of measurement. For each 
objective within each learning area, an answer is sought to the question: 
'Tfow much do we know?" But exercises designed with unequal difficulty 
for each objective are hard to weight without determining criteria for 
such weights . 
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The science assessment in July 1970 reported: "There is no 
summary figure which describes how well any one objective or sub- 
objective is being achieved- There are no norms against which to 
compare the percentages of success on individual exercises. This is to 
be expected since National Assessment was not designed to provide any 
of these . " 

By the time the reading assessments Preliminary Report (02-R-OO) 
came along in May 1972, the notion of reporting results by ''themes" had 
evolT'ed. National reading themes, developed by the staff of the National 
Assessment, fall into two loose clusters of difficulty. Median percentages 
are computed for er.ch theme and by objective. Medians for the nation in 
reading reportedly vary little for groups, with medians for each group 
defined as percentages of success which are midpoints in ranges op^ 
percentage correct responses of the exercises. 

In the ten subject matter areas, data will become available over 
time on the percentages of correct responses for a large number of ex- 
ercises which measure objectives determined by subject-matter specialists, 
educators 3 and lay persons. 

After a series of exercises have been administered, the findings 
can be analyzed and a weighting system designed to determine units of 
measurement based on difficulty of response, and the weights of "parcels" 
of knowledge. 

Summary measures have been computed for learning areas. Aggregative 
medians now available show the median findings for the total number of 
exercises in a subject area, and for differences between subgroup perform- 
ances and national performance. 

National median scores are presented in Table 1 by learning area 
and by age group. The medians, by age group, are not median scores for 
the same exercises, but include all the exercises given. Other data 
reported show : 

- Reading scores are higher than experts expected. 

- The edge in success of adults over 17-year olds in ricience appears 
mostly in exercises drawing on personal -experience : the edge for 
17-year olds is marked where formal learning is ber.ng "'rested. 

- Writing skills are generally deficient. The median score at age 
9 is 33 .0 percent for those in the Northeast, and 10 pti-^centage 
points lower in the Southeast. 

- Overlapping exercises , * those given to more than one age group . 
indicate gains between age 9 and age 13 in science vary between 
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* See Appendix Table 4 for a preliminary tabulation of number of exercises, 
type of exercise, and number of overlapping exercises. 
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exercises but are substantial enough in percentage points to 
suggest progress in learning. Similar gains are not as marked 
in the older ages. 

Comparisons between males and females given in Charts 1 and 2 show: 

- In reading and writing, females have the edge over males. In 
science female scores increasingly lag behind males from age 9 
to young adulthood. 

- A deterioration in relative knowledge of females takes place in 
older age groups. Females lose their edge over males in reading 
but retain it in writing. Female median reading scores at age 9 
show an advantage over males which drops progressively until young 
adult male scores surpass or are equal to thost of females. 
Whether these findings indicate decline as femalos grow older 
cannot be determined since different person? ai\i represented in each 
age group and the generations may differ in male/female discrepancy. 

What more could we learn? 

The findings on subgroup differences suggest where, and through 
what "parcels" of information, .more knowledge could be acquired. 

The. edge of Northeast over Southeast is shown in Table 2. The 
advantage persists even when no adjustment is made for the influences 
of variation by race. Race variations* show whites and "other minorities" 
with large advantages over blacks in medians for all subject areas. The 
largest advantage of nonblacks is in science, the smallest in citizen- 
ship (Table 3). 

Suburbs consistently outscore inner cities and other areas; 
rural communities also exceed the inner city (Table M^) . 

Greater equality of educational product by raising the disadvantaged 
regions, places, and persons to the median scores of places and persons 
with greater advantage, represents one possible quantification in answering 
the question: "What more can we learn?" Certainly, if 85 percent of 
persons in an age group answer correctly in the Northeast, a similar 
percentage of correct responses to an exercise might be obtained, with 
some change in schooling, in the Southeast. Other standards for subgroup 
comparisons can yield information on what more can be learned. To cite 
another example, i.^ achievements of blacks were equalized to whites, the 
national educational product would be increased by a quantifiable percentage 



* Data for the first year were reported for black and uonblack which 

included whites and other minorities. Data for the isecond and subsequent 
years were reported for v,/hite, black and other, which included other 
racial minorities. 
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The staff of the National Assessment for Educational Progress 
reports a new effort to achieve estimates of expected levels of perfor- 
liiance in individual learning areas • A group of science educators 
representing the National Science Teachers Association has undertaken 
to make the National Assessment data more meaningful to science 
educators by using expert judgment to set a standard of expected level 
of performance as a basis for evaluacing the results from the science 
assessment. 

Even a cursory examination of the individual exercises and the 
methods of arriving at those exercises points to perhaps their major 
contribution to education^ namely, to help gain more learning by new 
practices. The major contributions of National Assessment are 
essentially byproducts of the statistics gathering effort such as: 

- identification of gaps in knowledge for use in teacher training, 
curriculum development, textbook and other educational materials 
development. 

- creation of new test instruments that go beyond pad and pencil 
tests, but also include some innovative pad-pencil testing 
methods. 

- formulation of objectives and subobjectives in sufficient detail 
to give new direction to teacher training, curriculum development, 
and textbook preparation 

- suggestions for new teaching methods that come out of the new 
test instrumentation. 

Use of the National Asseasment data for understanding next steps' 
toward additional learning would be facilitated by cross-classifications 
of data to suggest answers to such questions as: "Are persons in the 
Northeast whose parents had no high school learning more (or less) than 
those in the Southeast whose parents had no high school." "Do black 
males have higher scores, for what types of ex. seises, than females who 
are black?" "Is there a dropping off of achievements for black adult 
females as well as for tho^^e who are white?" "Are the rural area-city 
differences in white male a :ihievements for those who reside in the 
Northeast greater (or less) than the differences in white male 
achievements in the Southeast? 

Changes over time are still additional data to be provided by 
National Assessment. 



Are we learning more over time? 
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When the second round o£/ exercises for a -sxibject area is given, 
time-phased data will become available. Two approaches to the question 
can be taken: one waits on the second round of questioning, the other 



16 - 



draws on age differences to point to possible improvements in learning. 
If young persons know more than older youth, or young adults, or 
evidence less racial separatism, for example, then perhaps we are 
making some progress. Similarly, if regional variations among young 
persons are less marked than for the older youth or young adults, 
then similarly more evenness among regions may be anticipated. 

The nondisclosure of exercises needed to insure that the next 
round of data collection is unbiased, impairs the immediate usefulness 
of the National Assessment as a guide to teacher training, curricxxlum 
development, and textbook preparation, and as policy-relevant information 
on potential learning, especially by target groups. (For the science 
area, the barriers of nondisclosure will shortly be lifted.) 

What kinds of knowledge did everyone (or nearly everyone) have? 

For which exercises were the correct responses uniformly high, 
e.g., a high percentage of males and females responded correctly and so 
did both minorities and whites? For which exercises were correct 
responses the same in city and famland in Northeast and Southeast? 
When findings point to uniformity of knowledge or large percentages of 
correct responses, how are the findings to he interpreted? In reporting 
the science findings the generalization is made that the more practical 
the inf ormairion^ the more uniform the correct responses, e.g., the rpore 
the learning is connected to daily living, the less the disparity 
betv;een black and white, and city and suburb Conversely > findings from 
the more conceptual exercises show greater disparity between black and 
white, and similarly greater differences between those whose parents had 
post-high school education, and those whose parents had no high school. 

Are reading scores a ^ood proxy? 

Findings from the National Assessment for Educational Progress 
cast doubt on the usefulness of reading scores as a single measurement 
tool. "Reading" scores have come to be used widely in evaluation of 
educat5.on programs. Are such scores a good proxy for all of educational 
progress? The answer that the National Assessment appears to suggest 
is ^^no*" The median scores for reading are hi^er than for other of 
the learning areas — in facty so much above writing medians > for example, 
as to caution against continued use of "reading" as the sole measurement. 
Of course, part of the difference may be attributed to difference in 
difficulty of the exercises between sxabject areas. The reading test may 
have hem relatively easier than the other tests. Percentage points 
crain in reading is not large from one age group to another, but a growth 
pattern may Jndicate a developmental advancp by age. (In some subject 
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areas there is a dropping off of the median value.) This finding 
should help to indicate the need for more comprehensive measuring tools 
than some recently used. 



Does the education of parents p^^e the same hurdle to educational 
attainment in each of the learning areas? 

The National Assessment findings on median scores by parental 
education in the five learning areas suggest an extra deficiency in 
writing competence of the child from deficiencies in English usage in 
the home. Median writii^g scores at age 9 are low relative to scores 
in other learning areas; but in no otner area is the score for those 
whose parents had no high school education only about half that of those 
whose parents had post-high school education. The difficulty of over- 
coming the deficiency is indicav^^d by the movement in percentage points 
while at school. Differences between those whose parents had post-high 
school and those whose parents had no high school education drop for 
writing and reading after a sharp rise from 9- to 13-year olds, but 
the edge for the young persons whose parents had post-high school education 
remains relatively stable in the case of science and literature. (Chart 3) . 



Is the retention of knowledge sufficient to indicate that existing 
methods of preparatory education are efficient? 



In a number of the exercises given to both adults and to younger 
persons, the adults had a smaller percentage of "correct" responses. 
Science information parr:eled out to children, in particular, may be 
forgotten in adulthood. Even in the writing exercises that overlapped, 
a declining success rate is reported. For example, in one identical 
exercise 13-year olds, 17-year olds, and young adults were asked to 
complete an application blank. Only 50 percent of the adults did so 
correctly while 61 percent of the i7-year olds coiald. When 17-year olds 
and adults were asked to write directions for making or doing something, 
three quarters of the 17 -year olds made acceptable responses compared to 
little over half of the adults. In general, adults failed to respond 
more often than those still in elementary and secondary school years, 
and reported "I don't know" more frequently. The National Assessment 
staff offers a number of explanatory factors: "loss of skill from lack 
of practice, less cooperative behavior, or improvements in school 
programs since the adults received their formal education. 
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On what types of exercises was there almost uniformly incorrect 
response? 



The question arises from the decline in "parcels" of knowledge 
in adulthood: "Should ma terial' subject to such decline be taught 
before adulthood as a part of preparatory education?" One argument 
for retaining such material lies in the potential accumulation of 
information in subsequent years of study. For the youjig person going 
on to university studies in the sciences, for example, the information 
on science in the lower and middle level are important to his 
identification of field of interest and selection of future studies. 
Has sufficient knowledge been acquired earlier to make a real 
difference in university studies? Do these effects counterbalance 
the loss of knowledge of many persons? In brief , could there be more 
efficient timing of knowledge transmission? 



III. AN AGENDA FOR THE FUTURE 



The National Assessment work, completed and projected, points 
to new approaches. Some of the follow-up studies would make more 
useful current National Assessment plans; others would enlarge the 
scope or deepen the data so that they may be applied more generally* 



Social Reporting as Part of National Assessment 



Social Indicator quantification suggests that consideration 
night well be given to enlarging and interrelating the information 
gathered by the National Assessment so that social reporting may be 
improved in this decade without large added costs. These steps can 
be taken: 



Assessment of possible add-ons to National Assessment exercises 
for revisions of such exercises) which measure social conditions 

Measurements of social conditions appropi^iate to national 
assessment processes are those that can be obtained by standard*- 
ized testing. As an initial step toward application of the tools 
of National Assessment to broader social reporting, measurements now 
developed and used for research purposes , such as measures of 
social psychological attitudes or political attitudes, should 
be examined carefully for inquiry into: 
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(a) What types of social conditions can be assessed by 
responses from individuals through a "testing" 
procedure? 

(b) What empirical instruments are at hand to measure 
social conditions and change? 

(c) Which of those instruments are useful in a survey 
research setting? 

(d) What would be gained by a recurring application of such 
instruments? 

(e) What additional research is indicated? 



New exercises on social conditions. 



The National Assessment machinery for objective setting and 
exercise developmen': might be examined to determine whether 
new machinery would be needed if social conditions were 
explicitly assessed as part of the ongoing program. What 
criteria might be used in such a determination? How might 
account be taken of (a) incremental costs, (b) achievement 
of educational purposes of x!:;tional Assessment such as 
curriculum development, teacher training and so forth, 
(c) achievement of the purposes of social assessments? 
Do overlapping purposes make feasible a composite set of 
criteria for priority in selection? For example, in science, 
questions that are scientific in nature could be included 
in the science exercises with precedence over others if, 
at the same time, they lend themselves to social reporting. 

Add-ons to existing survey machinery . 



Alternatively, the adaptation of National Assessment to social 
reporting might take the form of special survey exercises given 
to small samples of young persons. Thus the machinery of 
National Assessment might be used more economically to obtain 
the added social data without necessarily amending existing 
educational progress reporting. The device of a supplementary 
survey piggy-backed on a continuing sample survey is femiliar. 
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Improving Measures and Application of Measures of Educational Progress 



We ask also: "What is required to make the National Assessment 
quantify educational progress better than it now does?" This question 
i^s raised not to understate the usefulness of National Assessment data 
now becoming available, but rather to point to some areas of improvement 
and a deepening of the data base that would enhance its usefulness. 
Among the changes that might be considered are: 

- Broadening the assessment to approximate more closely the 
multi -dimensional facets of educational products. 

- Adding input data to facilitate analysis of educational 
products in relation to educational resources. 

- Moving toward an intergovernmental comparative system of 
assessment to serve better the purposes of the Interstate 
Compact. 

- Applying a "SIR index"* to improve intertemporal and interjuris- 
dictional comparisons . 

- Developing a composite index to facilitate educational policy 
applications of the National Assessment. 

- Enlarging the research component to (among other things) reduce 
costs; improve statistical methodology on sampling and testing; 
and analyze characteristics of place and person in relation to 
assessment findings. 

1. Multidiinensions of educational products . The long accepted view — 
that achievement testing is the only way to measure a child's success at 
school, and the school's success witrh the child — is giving way to a 
realization that more than cognitive learning develops the person. As 
a Public Services Laboratory report puts it,. "Discussions of evaluation 
and analysis of education are sometimes premised on the odd notion of an 
abstract cognitive man who has no emotions, motivations, values, attitudes 
interests, or personality characteristics." (15) 

Cognitive measures fall short of accounting for all variation in 
learning development and academic performance, accounting for 25-50 
percent of school performance. The basic error in limiting measurements 
to cognitive skills alone may be sharpened by drawing an analogy to 
measuring human size. What are the dimensions of humans, for example, 
for purposes of -determining what kinds of clothes to manufacture? Many 
variables account for the "fit" of clothes. The key lies not in trying 
to take all variables into account but rather in selecting those that 
account for a significant (and satisfactory) share of the variation in 

ERIC " 

* Sex, income, and race. See page 24. 
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"fit." The pair of variables that prove most satisfactory for size in 
clothes come from two different analytic sets, namely height and girth. 

How might this analogy be applied in measuring educational output? 
We would base measurements of educational product on the different analytic 
sets to get a more meaningful measure of learning development. Else- 
where we identified the 4 A^s of educational products: achievements, 
attributes, attitudes, and aptitudes . (16) These educational dimensions 
recjuire data beyond those now reported in National Assessment. 

A new^ process is required to extend and deepen the data to include 
attributes and a range of attitudes that may be less closely tied 
to subject matter than are the present exercises. Among the elements 
especially important ere self-esteem and internal -external control. James 
Coleman in his historic survey of Equality of Educational Opportunity, (17) 
reported on both self-esteem and se- je of control. Based on his measure- 
ment, self -concept and control "may be important both as a factor affecting 
their school achievement and as a consequence of school for achievement 
later in life." 

Steps also have been taken by the National Center for Educational 
Statistics to include measures of self-esteem and internal-external 
control in their longitudinal study of high school youth. Such 
measurements are likely to be included in the NCES early childhood 
education surveys. 

More research of several kinds appears desirable before a routine 
reporting system of high professional competence could be designed that 
would measure these factors. Particularly, earlier research that we have 
done suggests the following: 

Review of existing exercises that tesr for self-esteem and external 
and internal control. (An estimated 200 instruments measure self- 
esteem alone . ) 

Evaluation of findings from small-sample studies in which the various 
measures have been used. 

Conduct of small experiments with instrument testing, to establish 
th2 kinds of instruments that could be applied uniformly to large 
numbers of persons. 

Selection of instruments based on evaluation, review, and experi- 
mentation. 

Reassessment of instruments as educational product measures, taking 
account of the special uses of such instruments as a component of 
National Assessment. 

Design of agenda for developmental work on measurement concepts, 
standardization of testing instruments, formulation of large- 
scale application techniques. 



(a) 

(b) 
(c) 

(d) 
(e) 
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2. Pi^oxy Simplification and Selection . The world of knowledge is 
clearly not tapped by present exercises in National Assessment. The 
key to the future is not to take account of all variables or try t:o, 
but rather to select more knowledge ably than is possible at present among 
the proxies that would measure a significant share of the differences in 
learning in the population. 

As more and i.iore findings from National Assessment become avail- 
able, factor analysis may identify clusters of questions for which 
th "^indings are th ^ '^ane^ or nearly the same, indicating that one, or 
fet-er, exercises ma'^* be used as proxies for the remaining questions in a 
cluster. A systematic analysis of findings to explore substitutability 
ana improved selection seems indicated. In such an examination the 
similarity of findings from exercise to exercise for each of the 
characteristics such as sex, race, and city size would be considered 
separately. In this way the National Assessment processes might be 
simplified over t*ime. 



3 . Educational Resources and Educational Progress. . The question of what 
we know has a counterpart: 'Wiat resources did it take to learn what 
we know?" And, if sufficient numbers of us do not know even basic 
skills, what combination of resources is required to achieve the basic 
learning? 

The National Assessment program as now designed is in fact one 
of the three elements in the drive to educational accountability which 
has sustained interest in an assessment of educational progress. The 
other two elements are} data on resource inputs and on processes of 
combining those inputs . 

The National Assessment provides a unique opportunity to link 
input, process, and measurement of educational progress. Without 
intending to understate the many complex problems of an expansion of the 
existing characteristic data to include information about the schools 
(the teachers--their training and experience; facilities and equipment) 
and their finances, the future usefulness of the National Assessment for 
policy depends upon such exp^^nsion. The strong drive for accountability 
in education indicates a beginning to fashion the links between input 
and process data, and measurements of educational output. 



State Assessment Policy . Policy purposes require State by State 
educational product information. Data requirements for policy guidance 
have traditionally led the Office of Education to provide comparable 
data across States, serving both national purposes and program planning 
and implementation in the States and communities. 
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While the emphasis on educational products has created an urgent 
need for statistics on educational output for educational planning, so- 
far the FeJeral Government has not concerned itself with achievement 
measurement across State lines. The Educational Testing Service in 
collaboration with the Education Commission of the States and the 
Education Resources Information Center surveyed the State educational 
assessment programs in the fall of 1970 and again more recently. The 
purpose of the surveys was to find out about what States are doing 
with regard to State -wide educational assessment. These surveys 
suggest that, as of December 1972, only four States--Maine , Massachus'^tts, 
Connecticut and Iowa--have applied National Assessment exercises for 
State assessment. A number of States have moved^ as part of their 
program evaluation, to some achievement score reporting but not of the 
National Assessment type. Some 30 States are reported to be considering a 
change to National Assessment exercises.* 

So far, questions about State assessment of educational products 
have failed, it seems to me, to consider the possibilities of substituting 
for the National Assessment a truly intergovernmental system of 
achievement data collection drawing on experimental work and implementation 
done by the National Assessment. Among the factors to be considered are 
usefulness of State-by-State "need data" that incorporate information 
on educational products, and the techniques or feasibility of a building- 
block approach that would depend on uniformly defined modules of 
information that would .be standardized but would also leave room fzv 
flexibility in the States; for example, special learning area objectives. 
Economies ".n professional skill and tech^cal design would be part of the 
product of an intergovernmental effort, /as would interstate comparisons. 

The Federal Qovemment has an interest in assuring that its own 
tax dollars for education are well sp^nt. But findings on educational* 
progress ' ar e_espeoially -.i^TTpQ;rt^ SiSr State educational policies. All told, 
jthe Fe^^i^aTshare of resources for elementary and secondary education is 
relatively small — under 10 percent; and while the national leadership 
role is of great importance, immediate applications of assessment data 
are more likely to be made by States and communities. 

The various uses of educational product measures in the States 
include the following: 

- Indices for intrastate allocations of funds. 

- Indicators as guides for State appropriation decisions. 

- Descriptors of program requirements leading to urogram design. 

- Indices of effectiveness of teacher qualification requirements 
and training. 

* Since December 1972, Colorado has added Science assessment and eight other 
States are veil along in plans for assessments. 



r 



2'4 - 



- Quantifications for use in program analysis. 

- Benchmark indices for rest '^g aad. evaluating program policies 
and projects. 



5. A "SIR Index" of Educational Progress , For comparing educational 
progress across time, and across geographic boundaries, the statistical 
tool kit of the biostatistician and demographer might be used to gain 
an additional set of measurements, measurements calling for a statistical 
correction for population differences over time, and between places. 

Various processes of standardization for population differences 
have been worked out. Each of the processes essentially answers a 
different question. One question may be stated in this way: "If 
the population characteristics were the same in two time periods (or 
two places) would the average score in each be higher or lower than the 
scores that have not been adjusted by standardization for population; 
or is the difference in raw scores a statistical artifact reflecting only 
the particular population characteristics at one period of time or place?" 
Another question that standardization for populations characteristics 
would help answer is: "Is the average score in one place actually higher 
or lower than that in another, if population characteristics were the 
same for the two places?" 

We earlier formulated and proposed a "SIR" educational achievement 
index that could be applied comparatively. The "SIR Index" is intended 
to adjust to a normal or standard population for sex, income and race. 
In recent months the "SIR Index" has sometimes been confused with the 
balancing procedure of the National Assessment. However ^ the balancing 
procedure gives the achievemen: data for a group, for .example, whites, and 
seeks to exclude variations in such data about whites due to place, 
region, parental education, by a process that distributes the whites 
proportionately across the various characteristics^ groups. 

The "SIR adjusted index" in contrast, seeks to show not an adjusted 
score for whites, to continue the example, but the overall score adjust- 
ed for variation across time, or places in the distribution of population 
by race. If blacks, for example, become a larger share of the population 
in an age group and whites a decreasing share, the SIR woiild adjust the 
base of comparison so that the black-white ratio were the same in the two 
time periods. 

Furthermore, balancing is now applied to factors over which 
schools have control, while the "SIR adjusted index" is intended to 
control statistically for those population characteristics over which 
schools have no control, namely, sex, income, and race-* 
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ERJC * Adjustments for age convert the "SIR" adjustments to "SIRA." 
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5. Towards a Composite Index ^ Frequently it is proposed that National 
Assessment be applied as an educational measure comparable to Gross 
National Product (18) . These suggestions seem to me to be most 
inappropriate and strained. Gross National Product is an accounting term 
measuring, on the one hand, a dollar value product that combines 
measurements of quantity and quality of product and on the other, 
factor costs, or the composite cost of the inputs of \e various factors 
of production wages, rents, and interests In contrast. National 
Assessment is basically a quality measure of educational output without 
a price, or even a quantity component. 

The closest dollar coixnt of education would be of two kinds, 
neither of which has much to do with National Assessment. The first of 
these is a measure of educational inputs in dollars, another way of 
saying expenditures for education. Alternatively thee count could be 
somewhat less orthodox and include as part of the investment component 
of Gross National Product an educational measure, determined by each 
yearns present value of future earnings from school completions or 
terminations. 

As one reviews the intent underlying the suggestion for a composite 
index, it seems possible to retain this notion without the analogy to 
Gross National Product. I suggest for purposes of discussion an analogy 
of t) ^ educational product index to the consumer price index (or CPI) . 
This analogy is closer than CtV- in rrethodology and also in potential 
kinds of appl\cations . iJational Assessment data could be presented in 
terms either of change from a base year, or as a level amount. To 
follow th*" analogy in terms of method. National Assessment collects 
data (for a "basket" of exercises, related to a "basket" of learning 
areas). To arrive at a composite number, a weighting system is required 
with changes in weights as objectives for education and curricula change 
from time to time. CPI depends upon the pricing of a specific basket of 
goods of uniform standards and quality. Pro^:ies are used for each 
component of that basket of goods just as proxy exercises are clearly 
used in the National Assessment, Weights for the different components 
of the basket are obtained /Hy periodic (recently about once each decade) 
family surveys to determine changes in patterns. 

The EPI or educational product index derived by defining baskets 
of learning areas, and later weighted on the basis of special surveys 
(and in accord with difficulty or level of knowledge) would be an 
abstraction. The CPI is also such an abstraction that, nevertheless 
because of the highly professional development and assessment, conges to 
be widely used as a barometer of change, EPI also could bectme a widely 
used policy instrument. 

The analogy in use between CPI and EPI is even closer than the 
O ithodological analogy, CPI is used to signal price changes and becomes 
JCt index to determine the need for compensatory economic action, an 
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adjustment of expenditure data^ a guide to salary policies. Sijnila?:»ly ^ 
EPI could be used as a barometer of change in educational competences, 
as an index tro signal the need for compensatory action ^ and as an 
adjustment for expenditure data. Expenditures in two time periods 
per pupil might be equal but when corrected for EPI, the EPI adjusted 
figure might show a gain (or loss) ov^,:^ time. Equal expenditures do not 
mean equal quality of product. 

In the case of education, growth in expenditures may mean 
unchanged expenditure per unit of quality, or even a falling off 
of expenditures per quality unit. Or EPI might be used just as CPI 
is now--as a payment adjuster or as a way to better quantify need for 
enlarged fiscal resources applied to education. 

Many steps lie ahead before such an index could be designed and 
implemented. At present, a basket of learning areas has been* determined 
and proxies in the form of exercises have been selected. The new step 
is a weighting system. In addition , there lies ahead the rurther problem 
of taking account through proxies of a significant share of the variation 
in learning but with variables or exercises reduced to the minimal 
number required to carry out the purpose. 

7* More Research on Methodology and Measurement . Essentially a new 
measurement tool Is being forged by the National Assessment, It is not 
surprising that in a community that has long concerned itself primarily 
with such measurements as size of class or teaching unit^ teacher salary 
levels, numbers and proportions of experienced teachers and expenditures 
per pupil, sta-^,istics on educational progress in learning i-^equire support 
of mucli research and development. There is a long distance yet to go in 
analysis of initial findings, not only for subsequent economy in data 
collection, but also in understanding feedback consequences for 
methodology such as sampling procedures, adjustments of raw scores, and 
construction of a useful composite index of educational product. There 
is even further distance for those who would apply the information 
to policy. Lack of patience in applying new accountability tools has 
in my judgment already created a crisis; it is a cx^isis born of the 
current doubt that schools matter. It becomes especially urgent that 
the National Assessment measurements be designed with utmost professional 
skill, drawing on the best of the intellectual capacity of the n^::tion 
in determining what it is that it is important to Know, in order to 
assess what it is that is known. It becomes imperative that the 
findings analysed and presented in the best of the objective statis- 
tical tradition with appropriate cautions about error of sampliTig, and 
bias of findings from sample design. And it is especially urgent that 
adverse findings, along with those favorable to the educational commimity, 
be fully disclosed so that the public debate has the full measure of facts 
and a better understanding of the uncertainties when the decision is taken 
on resource allocation. 
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Accountability is a management concept that cannot be denied 
It requires, however, the best 6f quantification tools of a size ' 
appropriate to the size of the resource commitment. Small, hesitant 
steps toward measurement in the light of the current crisis in the 
schools would now deepen it. 

^->,-o ^""^.Z^^^ Dickens' Gradgrind. if we do not get to heaven 
tnis way. It IS r^oi- p socioeconomic place and we have no business 
there . 
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CHART 1 



Male-Female Edge on National Assessment Tests* 
By Subject and Age 



Percentage 
Points Difference 
12 



10 



4 - 



6 - 



8 - 



10 - 



12 



Age 



Science 



(9) 



Writing 



(13) (17) y 

Citizenship ^^Jff 



(A) 
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For values, see Table 5. 



' Unpublished Tables of "Median Delta P's and P's by Category," prepared by 
National Assessment of Educational Progress, October 12, 1972. 
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CHART 2 

Median Male- Female Differences 
for 

Physical and Biotogicsi Science Exercises 
of the 
National Assessment* 



Percentage 

Points 

Difference 

15|- 



Physical 
Science 
Exercises 



10 




EKLC 



•NAEP, Group Results A 1S71, Report 4, pg. 8. 
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CHART 3 

Edge for Those Whose Parents had Post High School Education 
Over Those with No High School Education 
(Percentage Points) 
Science 3q Citizenship 



% 20- 



21.5 



22.6 



17.2 



22.1 



10- 



Age 9 ^2 17 Adult 



% 20- 



15.8 



-17.9 



21.8 



20.2 



10- 



Age 9 13 17 Adult 



3C--. 



Reading 



% 20- 



18.9 



24.6 



20.0 



16.0 



10- 



Age 9 13 17 Adult 



30 



Literature 



30-1 



Writing 



% 20- 



15.3 



23.3 



10- 



22.8 



20.3 



Age 9 13 17 Adult 



% 20- 


















18.8 


19.3 




20.9 


10- 




22.7 



Age 9 13 17 Adult 



For values, See Table 6. 



r 



- 33 



30-1 



CHART 4 

White Edge - By Subject and Age* 

■ (Percentage Points) 



% 20- 



25.0 



10 



25.5 



19.2 



25.2 



Age 9 13 17 .'^duit 



% 20- 



10- 



Citizenship 



15.0 



15.7 



20.1 



16.2 



Age 9 13 17 Adult 



30-1 



Reading 



% 20- 



21,1 



10- 



Age9 13 17 Adult 



30-1 



Writing 

















% 


24.7 


21.5 


23.3 





10- 



19.3 



23.5 



18.5 



20.1 



Age 9 13 17 Adult 



30 



Literature 



% 20- 



10- 



17.6 



19.0 



Aged 13 



16.1 



24.2 



17 Adult 



ERIC 



For values, see Table 3. 



^Over Blacks at same age for sane subject. 
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TABLE 1 

National Median Percent Acceptable Answers on 
National Assessment Exercises* 



Age 

Siibject 9 13 17 Adult 



Literature 


1+4.9 


53.3 


61.1 


63.0 


Writing 


28.3 


55. M- 


62.5 


58.4 


Reading 


72.5 


71.8 


80.3 


83.4 


Citizenship 


64.1 


63.2 


61.8 


60.5 


Science 


68.3 


58.4 


47.1 


51.3 



* National Assessment of Educational Progress. Subject Papers 
(Literature, Writing, Reading, Citizenship, Science), Median 
Delta P's and P's by Category. October 1972. Processed. 
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TABLE 2 



Median Percent Acceptable Answers 
For the Northc=ist and Southeast Regions 
on National Assessment Exercises* 



Age 



Subject 



13 



17 



Adult 



Science 
N.E. 
S.E. 



71.6 
60.9 



62.2 
4^9.6 



1+9.1 
39.9 



53.6 
1+2.7 



Citizenship 
N.E. 
S.E. 



65.4 
60.8 



65.5 
59.3 



61+.5 
50.9 



63.9 
53.3 



Reading 
N.E. 
S.E. 



76.2 
65.1+ 



71+.9 
65.6 



82.8 
71+.6 



83.9 

7i+.ii- 



Writing 
N.E. 
S.E. 



33.0 

23,9 



60.1+ 
50.9 



61+.9 
51+.6 



61.0 
51.1+ 



Literature 
N.E. 
S.E. 



1+6.3 
37.5 



51+.8 
1+7.0 



62.8 
56.3 



65.9 
52.9 



* NationetL Assessment of Edaca-*- zonal Progress. Subject Papers 
(Literature,^ Writing, Readii:g, Citizenship, S-'ience), Median 
Delta P's and P's by Category. October 1972. Processed. 
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TABLE 3 

Median Percent Acceptable Answers by Race 
on National Assessment Exercises* 



Age 



Subj ect 


9 


13 


17 


Aduli 


Science 










White 


72.7 


62.6 


48.9 


54.4 


Black 


47.7 


37.1 


29.7 


29.2 


Other 


52.6 


45.0 


37.2 


43.4 


Citizenship 










White 


66.5 


65.8 


65.7 


63.3 


Black 


51.5 


50.1 


45.6 


47.1 


Other 


51.0 


U7.8 


49.9 


50.6 


Reading 










White 


75.4 


76.0 


82.6 


85 „ 7 


Black 


5tf.3 


51.3 


61.1 


62.4 


Other 


60.5 


67.1- 


76.0 


74.5 


Writing 










White 


31.6 


60.7 


65.2 


60.9 


Black 


12.3 


37.2 


1:6.7 


40.8 


Other 


15.6 


39.0 


53.0 


45.5 


Literature 










White 


18.1+ 


56.2 


62.8 


66.1 


Black 


30.8 


37.2 


46.7 


41.9 


Other 


33.8 


50.4 


53.8 


55.9 



* National Assessment of Educational Progress. Subject Papers 
(Literature, Writing, Reading, Citizenship, Science), Median 
Delta P's and P's by Category. October 1972. Processed. 
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TABLE 4 



Median Percent Acceptable Answers 
For Rural and Jnner-City Areas 
on National Assessment Exercises* 



Age 



Siibject 



13 



17 



Adult 



Science 
Rural 

Inner City 



57.8 



47.5 
39.1 



37.3 
35.6 



45.2 
34.5 



Citizenship 
Rural 
Inner City 



54.9 
49.5 



55.8 
52.5 



54.6 
53.1 



57.3 
55.3 



Reading 
Rural 

Inner City 



67.0 
56.6 



66.2 
60.5 



76.6 
7?. 4 



N.A.** 
N.A. 



Writing 
Rural 
Inner City 



21.5 
13.3 



47.1 
41.4 



59.7 
44.8 



52.3 
45.7 



Literature 
Rural 
Inner City 



39.5 
34.1 



51.1 
45.4 



59.4 
52.4 



N.A. 
N.A. 



* National Assessment of Educational Progress. Subject Papers 
(Literature, Writing, Reading, Citizenship, Science), Median 
Delta P's and P's by Category. October 1972. Processed. 



** N.A. - Not available. 



- 38 - 



TABLE 5 

Median Percent Acceptable Answers 

For Males and Female." 
on National Assessment Exe^-cises* 



Age 



Subject 



13 



17 



Adult 



Science 
Male 
Female 



70.1 
67.5 



60.8 
56.1 



51.2 
W.6 



57.9 
1^5.8 



Citizenship 
Male 
Female 



63.7 
53.8 



62.8 
62. S 



63.3 
62.3 



64-. 0 
59.0 



Reading 
Male 
Female 



69.1 
75.8 



69.1 
74-. M- 



78.5 
82.6 



83.0 
82.1 



Writing 
Male 
Female 



22.3 
3tf.7 



51. ^ 
60.9 



57.3 
67.0 



50. M- 
62.3 



Literature 
Male 
Female 



M-3.5 
'^M-.l 



50.3 
53.0 



59.8 
61.1 



63.0 
63.3 



* National Assessment of Educational Progress. Subject Papers 
(Literature, Writing, Reading, Citizenship, Science), Median 
Delta F's and P's by Category. October 1972. Processed. 
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TABLE 6 

Median Percent Acceptable Answers 
on National Assessment Exercises 
By Parental Education — Post High School 
Education and No Higli School Education * 



Age 

Subject , 9 13 17 Adult 



Science 

Post H.S. 76.7 66.6 52.2 64-. 0 

No H.S. 55.2 W.O 35.0 41.9 

Citizenship 

Post H.S. 68.7 64.1 66.6 ''3.2 

No H.S. 52.9 46.2 44.8 53.0 

Re ading 

Post H.S. 80.0 80.1 86.9 90.8 

xSIo H.S. 61.1 55.5 56.9 74.8 

Writing 

Post H.S. 35.1 55.1 70.1 67.9 

No H.S. 19.8 41.8 47.3 47.6 

Literature 

Post H.S. 51.5 60.5 69.3 74.1 

No H.S. 32.7 41.2 46.6 53.2 



* National Assessment of Educational Progress. Subject Papers 
(Literature, Writing, Reading, Citizenship, Science), Median 
Delta P's and'P's by Category. October 1972. Processed. 
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TABLE 7 



Reading Achievement by Age and by Theme 
(Median Percenta<Tes Successful) 
197C1-1971* 



Subject 



13 



Age 



** 



17 



Adult 



Word Meanings 87 

Visual Aids 85 

Written Directions 81 

Reference Materials 6M- 

Facts from Passages 60 

Main Ideas from 

Passages 45 

Drawing Inferences 

frcHTi Passages 78 

Critical Reading 58 



76 
72 
83 
74. 
71 

51 

59 
60 



68 
81^ 
81^ 
85 
81^ 

68 

68 
72 



72 
80 
86 
93 
88 

75 

50 
70 



* An>erican Education, August-September 1972. "Statistic of the Month: 
Reading as an Indicator of Educational Attainment." 

** Only a fraction of the questions were the same for different age 
groups. The medians may therefore reflect different levels of 
difficulty. 
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Appendix Table 1 — Sample Population Categories 



Characteristic 
Sex 

Region 
Color 

Size-and-Type of Community 
Level of Parental Education 



Group 



- Male 

- Female 

- Northeast 

- Southeast 

- Central 

- West 

- Black 

- White 

- Other 

- Rural 

- Inner City 

- Affluent Suburb 

- Inner City Fringe 

- Suburban Fringe 

- Medium City 

- Small City 

- No High School 

- Some High School 

- Graduated from High ^lOol 

- Post High School 



r 
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Appendix Table 2 ~ Timetable for Assessment Exercises* 



Cycle 1 



March 1969 - February 1970 
October 1970 - August 1971 
October 1971 - August 1972 
October 1972 - August 1973 
October 1973 - August 1974 

October 1974 - August 1975 



Science, Writing, Citizenship 

Reading, Literature 

Music, Social Studies 

Math, Scier.ce** 

Wr? Jing,** Career and Occupa- 
tional Development 

Citizenship,** Art 



Cycle 2 



October 1975 
October 1976 
October 1977 
October 1978 



August 1976 
August 1977 
Augusv 1978 
August 1979 



October 1979 - August 1980 



Reading, Literature 
Music, Social Studies 
Math, Science 

Career and Occupational Develop- 
ment, Writing 

Citizenship, Art 



* National Assessment of Educational Progress^ Questions and Answers ^ 
Education Corronission of the States, April, 1972. 



** Subjects tested twice in Cycle 1 and a third time in Cycle 2. 
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Appendix Table 3 — Educational Objectives in 
Five Subject Areas* 



Subject 
Reading: 



Career and Occupational 
Development : 



Citizenship: 



Objectives 



Comprehend what is read 
Analyze what is read 
Use what is read 

Reason logically from what is read 
Make judgments coicerning what is 
read 

Have attitudes about and an interest 
in reading 



Prepare for making career decisions 
Improve career and occupational 

capabilities 
Possess skills that are generally 

useful in the world of work 
Practice effective work habits 
Have positive attitudes toward work 

Show concern for the welfare and 

dignity of others 
Support rights and freedoms of all 

individuals 
Help maintain law and order 
Know the main structure and functions 

of our government 
Seek community improvement through 

active, democratic participation 
Understand problems of international 

relations 

Support rationality in commtrnication, 
thought and action on social pro- 
blans 

Take responsibility for own personal 
develojmient and obligations 

help and respect -dieir own families 
(ages 9, 13, 17) 

Nurture the development of their 

children as future citizens (Adults) 
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tt^ijpendia Table 3 — Ljntinved 



Writing: - Write to communicate adequately in 

a social situation 

- Write to cOTmtunicate in a business 

or vocational situation 

- Write to coTcfnunioate adequately in 

a scholastic situation 

- Appreciate the vnlue of writing 

Sciencse: - Know fundamental facts and principles 

of science 

- Possess the abilities and skills 

needed to engage in the process 

- of science 

- Understand the investigative nature 

of science 

- Have attitudes about and apprecia- 

tions of scientists, science and 
the consequences of science that 
stem from adequate understanding?. 



* NatiOi^al Assessment of Educational ProCTfiss ; 

Si^ ience Objectives . 1969; Writing- Objective s. 1969; Citizenship 
Objectives, 1969; Reading Objectives . 1970; Career and OQCUpational 
Development Objectives . 1971; Education Commission of the States, 
Denver^ Color ada. 

ERIC 
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Appendix Table i}- Number of Exercises by Type* 



Year ui bcience 




Year 01 Citizenshio 




Age y : 




Age 9: 




Total Number: 


1 ii c 


Total Number: 




Ntunber Group : 




Numbej? Group: 


3 


Number Individual i 




Number Indiviaual: 


i+G 


Nimiber Unique: 


120 


Number Special: 


1 


Number Overlap: 


25 


Number Unique: 


22 


Number Released: 


59 


Number Overlap: 


22 






Number Released: 


21 


Age 13: 












Age 13: 




Total Number: 


lie 

lib 




mUTiDer broup • 




Total Number: 


78 


Number Individual : 




Number Group: 


27 


iNumoer unic]ue* 


Oj 


Number Individual : 


50 


Number Overlap: 


53 


Number Special: 


1 


Number Released: 




Number Unique: 


15 






Number Overlap: 


63 


Age 17: 




Number Released: 


39 


Total Number: 


121 


Age 17: 




Number Group : 


TIT 
11 / 




Numoer inaivmuaj.. 


Hit i 


Total Number: 


68 


Number Unique: 


5h- 


Number Group: 


37 


Number Overlap: 


67 


Number Individual: 


30 


Number Released: 


53 


Number Special: 


1 






Number Unique: 


3 


Age Adult: 




Number Overlap: 


65 






Number Released: 


33 


Total Number: 


116 






Number Individual: 


116 


Age Advilt: 




Number Unique: 


19 




Number Overlap: 


97 


Total Number: 


87 


Number Released: 


48 


Number Individual: 


87 






Number Unique: 


17 






Number Overlap: 


70 






Number Released: 


in 



* Two divisions of exercises: Those conducted in jgroups and those 
completed individually . 

Two classifications of exercises: Those unique and those over- 
lapping in objective or content with other exercises. 




*♦ Only 1 of 3 Science Administered per Respondent* 



Year 01 Writing 




Age 9: 




Tr>'t"nl XhimVioTi* 
XVLciX INUUlUJcl • 




1 r> VI 1 VI 1 ^ * 

ivurnoej? uroup. 


1 Q 


iNLUlUJcX XX1U.XV XULJ-CLX • 


■J 






NuTTiber Overlap: 




Number Released: 


12 


Age 13: 




XU Ld± INUJTlDcX . 


2fi 


All iniVki J >i ri>ioiir^ • 

i.Nijfi'Ot:ir uroup . 


1 7 


iNumoGr JLncii V iciiici-L . 


q 


Nuniber Unique: 


13 


Number Overlap: 


13 


Number Released: 


m- 


/*ge ± / : 




iorai iNuniDer: 


1 Q 


Number Group: 


1 Q 


Number Unique: 


7 


Number Overlap: 


12 


Numb t X? Released: 


1 n 


Age /VduLLt: 




iorax JNumJDer. 


1 fi 

i-O 


iNumDc r xnciivicru.cia. • 




jNumjjer unique * 


Q 


iNUJIlDGr vJVcr-Ld.p • 


o 
o 


INUJILDer XvcJ-cdScU. , 


1 n 


Year 02 Reading 




Age 9: 




Total Number: 


lL^o 


Nuniber Group: 


135 


Nuniber Individual: 


5 


Nuniber Unique: 


76 


Nuniber Overlap: 


64- 


Number Released: 


57 
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ArrO 1 -I • 




lo Ldx iNuniDe r • 


1 73 
1/0 


Number Group : 


X/U 


Number Individual: 


3 


Number Unique: 


22 


Number Overlap: 


151 


Number Released: 


78 


Age X / . 




iOLci± iNumDer* 




Number Group: 


132 


Number Individual: 


2 


Number Unique: 


10 


Number Overlap: 




Number Released: 


56 


Age -HClUa.L: 




1 o irai jNumD e r : 


c n 


ivjumuer xnuivxuua.± . 


DU 


Number Unicrue : 


n 
U 


Number Overlap: 


50 


Number Released: 


26 


Year 02 Literature 




Age y • 




lOLdJ- INLuTlijer. 


01 


j>iii]Tuj<j£' uroup. i . 


OK 


Number Individual : 


ii 


Number Unique: 


17 


Number Overlap: 


20 


Number Released: 


26 


Age 13: 




Total Number: 


51 


Nujnber Group: 


M-7 


Number Individual : 


Il- 


Number Unique: 


ls 


Number Overlap: 


38 


Number Released: 


33 



- L^8 



Year 02 Literature 




^ Year 03 Music 




Ajie 1 7 : 




Age Adult: 




Total Number: 


^8 


Total NiimHpT' 


Qli 


Number Group: 


111 


Numbpr Nnn— PptFot— 


Number Individual: 


7 


mance : 




Number Unique: 


3 


Number Performanoe: 




Number Overlap: 


L|5 


Number Unique: 


0 


Niimber Released: 


35 


Number Overlap: 




Age Adult: 




Year 03 Social Studies 




Total Number: 








Number Individual: 






Number Unique: 


0 


To'hal Numbpi?* 


/t7 
■r/ 


Number Overlap: 




Number Group: 


25 


Number Released: 


29 


Nurrber Individual: 


22 


Year 03 Music 




Number Unique: 


t^5 




Number Overlap: 


2 


Affe ^ : 




Arro 1 ^ • 




■HHII Nvunber: 


6M- 


Tn+"al NunibpT* 


Q ^ 


^^^^H;er Group: 


^7 


Number GrouD : 




i^fl^^er Individual: 


17 


Numbpr Individual : 


IQ 

X 3 


^Wumber Unique: 


13 


Number Unique: 




Number Overlap: 


51 


Number Overlap: 


89 


Age 13: 








Total Number: 


100 






Number Group: 


80 




Number Individual: 


20 


Total Number* 


X Xiy 


Number Unique: 


2 


Number Group : 


90 


Number Overlap: 


98 


Number Individu;*!: 


20 


Age 17: 




Number Unique: 


1 




Number Overlap: 


109 


Total Number: 


103 


Age Adult: 




Number Group: 


83 




Number Individual: 


20 


Total Number: 


101^ 


Number Unique: 


0 


Nimiber Non-Perfor- 




A. umber Overlap: 


103 


memce: 


52 






Number Perfonnance: 


52 






Number Unique: 


0 






Number Overlap: 


lOit 
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Year 04 Mathematics 




Year 04 Science 




Age 9 : 




Age 9: 




iorai Numoer: 


112 


Total Number: 


161 


Number Group: 


101 


Nurriber Group : 


137 


Number Individual: 


11 


Number Individual: 


24 


Number Unique: 


49 


Number Unique: 


94 


Number Overlap: 


63 


Number Overlap: 


67 


Age li: 




Age 13: 




iorai Numoer: 


162 


Total Number: . 


166 


Number Group: 


148 


Number Group : 


148 


Number Individual: 


14 


Number Individual: 


18 


Number Unique: 


5 


Number Unique: 


42 


Number Overlap: 


157 


Nuiriber Overlap: 


124 


Age 17: 




Age 17: 




Total Number: 


194 


Total Number: 


151 


Number Group: 


180 


Number Group: 


137 


Number Individual: 


14 


Number Individual: 


14 


Number Unique: 


21 


Number Unicjue; 


50 


Number Overlap: 


173 


Number Overlap: 


101 


Age Adixlt: 




Age Adult: 




Total Ntmiber: 


134 


Total Number: 


81 


Number Non-Perfor- 




Number Non-Perfor- 




mance' 


121 


mance: 


70 


Number Performance: 


13 


Number Performance: 


11 


Number Unique: 


6 


Number Unique: 


5 


Number Overlap: 


128 


Number Overlap: 


76 
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